System and method for aligning images

ABSTRACT

The invention is a system or method for aligning images (the “system”). A definition subsystem, including a first image, a second image, one or more target reference points, one or more template reference points, and a geometrical object. The definition subsystem identifies one or more target reference points associated with the first image and one or more template reference points associated with the second image by providing a geometrical object for positioning the first image in relation to the second image. A combination subsystem is configured to generate an aligned image from the first image and second image. An interface subsystem may be used to facilitate interactions between a user and the system.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.10/630,015, filed on Jul. 30, 2003, which application is herebyincorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

The present invention relates in general to the alignment of images.More specifically, the present invention relates to a system or methodfor aligning two or more images (collectively “alignment system” orsimply the “system”).

Image processing often requires that two or more images from the samesource or from different sources be “registered,” or aligned, so thatthey can occupy the same image space. Once properly aligned to the sameimage space, images can then be compared or combined to form amultidimensional image. Image alignment can be useful in manyapplications. One such possible application is in medical imaging. Forexample, an image produced by magnetic resonance imaging (“MRI”) and animage produced by computerized axial tomography (“CAT” or “CT”)originate from different sources. When the images are overlaid,information acquired in relation to soft tissue (MRI) and hard tissue(CT) can be combined to more accurately depict an area of the body. Thetotal value of the combined integrated image can exceed the sum of itsparts.

Another possible application of image alignment is for quality assurancemeasurements. For example, radiation oncology often requires imagetreatment plans to be compared to quality assurance films to determineif the treatment plan is actually being executed. There are alsonumerous non-medical applications for which image alignment can be veryuseful.

Several methods are available for image alignment, including automatedand manual alignment methods. However, currently available imagealignment tools and techniques are inadequate. In many instances,computer automated methods are unsuccessful in aligning images becauseboundaries are not well defined and images can be poorly focused.Although automated alignment methods perform alignment activities morequickly than existing manual alignment methods, manual alignment methodsare often more accurate than automated methods. Thus, manual imagealignment methods are often used to make up for deficiencies andinaccuracies of automated alignment methods. However, existing manualalignment systems and methods can be tedious, time consuming, and errorprone. It would be desirable for an alignment system to perform in anefficient, accurate, and automated manner.

SUMMARY OF THE INVENTION

The invention is a system or method for aligning images (the “system”).A definition subsystem, including a first image, a second image, one ormore target reference points, one or more template reference points, anda geometrical object. The definition subsystem identifies one or moretarget reference points associated with the first image and one or moretemplate reference points associated with the second image by providinga geometrical object for positioning the first image in relation to thesecond image. A combination subsystem is configured to generate analigned image from the first image and second image. An interfacesubsystem may be used to facilitate interactions between users and thesystem.

The alignment system can be applied to images involving two, three, ormore dimensions. In some embodiments, an Affine transform heuristic isperformed using various target and template points. The Affine transformcan eliminate shift, rotational, and magnification differences betweendifferent images. In other embodiments, different types of combinationheuristics may be used.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain embodiments of present invention will now be described, by wayof examples, with reference to the accompanying drawings, in which:

FIG. 1 is an environmental block diagram illustrating an example of animage alignment system accessible by a user.

FIG. 2A is a subsystem-level block diagram illustrating an example of adefinition subsystem and a combination subsystem.

FIG. 2B is a subsystem-level block diagram illustrating an example of adefinition subsystem, a combination subsystem, and an interfacesubsystem.

FIG. 2C is a subsystem-level block diagram illustrating an example of adefinition subsystem, a combination subsystem, an interface subsystem,and a detection subsystem.

FIG. 3 is a flow diagram illustrating an example of how the systemreceives input and generates output.

FIG. 4 is a flow diagram illustrating an example of facilitating apositioning of images and generating an aligned image according to thepositioned images.

FIG. 5 is a flow diagram illustrating an example of steps that an imagealignment system or method may execute to generate an aligned image.

FIG. 6 is a flow diagram illustrating an example of steps that a user ofan image alignment system may perform to generate an aligned image.

FIG. 7A is a diagram illustrating one example of target reference pointsassociated with a first image.

FIG. 7B is a diagram illustrating one example of a geometrical objectconnecting target reference points associated with a first image.

FIG. 7C is a diagram illustrating an example of a geometrical object anda centroid associated with that geometrical object.

FIG. 7D is a diagram illustrating a geometrical object and varioustemplate reference points positioned in relation to a second image.

DETAILED DESCRIPTION I. Introduction of Elements and Definitions

The present invention relates generally to methods and systems foraligning images (collectively an “image alignment system” or “thesystem”) by producing an aligned image from a number of images and arelationship between various reference points associated with thoseimages. A geometrical object can be formed from selected referencepoints in one image, copied or transferred to a second image, andpositioned within that second image to establish a relationship betweenreference points.

The system can be used in a wide variety of different contexts,including medical applications, photography, geology, and any otherfield that involves the use of images. The system can be implemented ina wide variety of different devices and hardware configurations. A widevariety of different interfaces, software applications, operatingsystems, computer hardware, and peripheral components may beincorporated into or interface with the system. There are numerouscombinations and environments that can utilize one or more differentembodiments of the system. Referring now to the drawings, FIG. 1 is ablock diagram illustrating an example of some of the elements that canbe incorporated into an image alignment system 20. For illustrativepurposes only, FIG. 1 shows a human being to represent a user 22, acomputer terminal to represent an access device 24, a GUI to representan interface 26, and a computer tower to represent a computer 28.

A. User

A user 22 can access the system 20 through an access device 24. In manyembodiments of the system 20, the user 22 is a human being. In someembodiments of the system 20, the user 22 may be an automated agent, arobot, a neural network, an expert system, an artificial technologydevice, or some other form of intelligence technology (collectively“intelligence technology”). The system 20 can be implemented in manydifferent ways, giving users 22 a potentially wide variety of differentways to configure the processing performed by the system 20.

B. Access Device

The user 22 accesses the system 20 through the access device 24. Theaccess device 24 can be any device that is either: (a) capable ofperforming the programming logic of the system 20; or (b) communicatinga device that is capable of performing the programming logic of thesystem 20. Access devices 24 can include desktop computers, laptopcomputers, mainframe computers, mini-computers, programmable logicdevices, embedded computers, hardware devices capable of performing theprocessing required by the system 20, cell phones, satellite pagers,personal data assistants (“PDAs”), and a wide range of future devicesthat may not yet currently exist. The access device 24 can also includevarious peripherals associated with the device such as a terminal,keyboard, mouse, screen, printer, input device, output device, or anyother apparatus that can relay data or commands between a user 22 and aninterface 26.

C. Interface

The user 22 uses the access device 24 to interact with an interface 26.In an Internet embodiment of the system 20, the interface 26 istypically web page that is viewable from a browser in the access device22. In other embodiments, the interface 26 is likely to be influenced bythe operating system and other characteristics of the access device 24.Users 22 can view system 20 outputs through the interface 26, and users22 can also provide system 20 inputs by interacting with the interface26.

In many embodiments, the interface 26 can be describe as a combinationof the various information technology layers relevant to communicationsbetween various software applications and the user 22. For example, theinterface 26 can be the aggregate characteristics of a graphical userinterface (“GUI”), an intranet, an extranet, the Internet, a local areanetwork (“LAN”), a wide area network (“WAN”), a software application,other type of network, and any other factor relating to the relaying ofdata or commands between an access device 24 and a computer 28, orbetween a user 22 and a computer 28.

D. Computer

A computer 28 is any device or combination of devices that allows theprocessing of the system 20 to be performed. The computer 28 may be ageneral purpose computer capable of running a wide variety of differentsoftware applications or a specialized device limited to particularfunctions. In some embodiments, the computer 28 is the same device asthe access device 24. In other embodiments, the computer 28 is a networkof computers 28 accessed by the accessed device 24. The system 20 canincorporate a wide variety of different information technologyarchitectures. The computer 28 is able to receive, incorporate, store,and process information that may relate to operation of the imagealignment system 20. The computer 28 may include any type, number, form,or configuration of processors, system memory, computer-readablemediums, peripheral devices, and operating systems. In many embodiments,the computer 28 is a server and the access device 24 is a client deviceaccessing the server.

Many of the processing elements of the system 20 exist asrepresentations within the computer 28. Images to be aligned by thesystem 20, such as a first image 30 and a second image 32, are examplesof processing elements existing as representations within the computer28. An image may include various reference points, and those referencepoints can exist as representations within the computer 28. Ageometrical object 35 of reference point(s) that is used to align afirst image 30 with respect to a second image 32, also exist asrepresentations within the computer 28.

E. Images

The images 30 and 32 can be any representation that can be read or actedupon by the computer 28, including graphical or data representations.The representations can involve two-dimensional, three-dimensional, oreven greater than three-dimensional information. One or more of theimages may be a digital image. An aligned image 38 can be formed fromany number of images.

An image is potentially any visual representation that can potentiallybe aligned with one or more other visual representations. In manyembodiments, images are captured through the use of a light-basedsensor, such as a camera. In other embodiments, images can be generatedfrom non-light based sensors or the sources of information and data. Anultrasound image is an example of an image that is generated from anon-light based sensor.

The images processed by the system 20 are preferably digital images. Insome embodiments, the images are initially captured in a digital formatand are passed unmodified to the system 20. In other embodiments,digital images may be generated from analog images. Various enhancementheuristics may be applied to an image before it is aligned by the system20, but the system 20 does not require the performance of suchpre-alignment enhancement processing.

The computer 28 may act upon multiple images in myriad ways, includingthe execution of commands or instructions that are provided by the user22 of the system 20. For example, the computer 28 can receive input fromthe user 22 through the interface 26 and from a first image 30 (a“target” image), a second image 32 (a “template image”), targetreference points 34, a geometrical object 35, and template referencepoints 36 generate an aligned image 38.

F. Reference Points

A reference point is a location on an image that is used by the system20 to align the image with another image. Reference points may be assmall as an individual pixel, or a large constellation of pixels. In apreferred embodiment, the user 22 identifies the reference points andthe system 20 generates the aligned image 38 from the reference pointsin an automated fashion without further user 22 intervention.

As seen in FIG. 1, target reference points 34 are associated with thefirst image 30 (the “target image”) and template reference points 36 areassociated with a second image 32 (the “template image”). Any number oftarget images can be aligned with respect to a single template image.The target reference points 34 and template reference points 36 arelocations in relation to an image, and the system 20 uses the locationsof the target reference points 34 and the template reference points 36to determine a relationship so that an aligned 38 can be generated.Locations of the template reference points 36 may be determined bypositioning the geometrical object 35 within the second image 32. Thus,the geometrical object 35 can be used to facilitate a generation of analigned image 38. In the embodiment shown in FIG. 1, a geometricalobject 35 is transmitted or copied from a first image 30 to a secondimage 32. In alternative embodiments, the geometrical object 35 may bereproduced in the second image 32 in some other way.

G. Geometrical Object

The geometrical object 35 is the configuration of target referencepoint(s) 34 within the target image 30 that are used to align the targetimage 30 with the template image 32. In a preferred embodiment, thegeometrical object 35 is made up at least three points.

II. Subsystem-Level Views

The system 20 can be implemented in the form of various subsystems. Awide variety of different subsystem configurations can be incorporatedinto the system 20.

FIGS. 2A, 2B, and 2C illustrate different subsystem-level configurationsof the system 20. FIG. 2A shows a system 20 made up of two subsystems: adefinition subsystem 40 and a combination subsystem 42. FIG. 2Billustrates a system 20 made up of three subsystems: the definitionsubsystem 40, the combination subsystem 42, and an interface subsystem44. FIG. 2C displays an association of a four subsystems: the definitionsubsystem 40, the combination subsystem 42, the interface subsystem 44,and a detection subsystem 45. Interaction between subsystems 40-44 caninclude an exchange of data, algorithms, instructions, commands,locations of points in relation to images, or any other communicationhelpful for implementation of the system 20.

A. Definition Subsystem

The definition subsystem 40 allows the system 20 to define therelationship(s) between the first image 30 and the second image 32 sothat the combination subsystem 42 can create the aligned image 38 fromthe first image 30 and the second image 32.

The processing elements of the definition subsystem 40 can include thefirst image 30, the second image 32, the target reference points 34, thetemplate reference points 36, and the geometrical object 35. The targetreference points 34 are associated with the first image 34, and thetemplate reference points 36 are associated with the second image 32.The target reference points 34 may be selected through an interfacesubsystem 44 or by any other method readable to the definition subsystem40. The definition subsystem 40 is configured to define or create thegeometrical object 35.

In one embodiment, the definition subsystem 40 generates the geometricalobject 35 by connecting at least the subset of target reference points34. The definition subsystem 40 may further identify a centroid of thegeometrical object 35. In addition, the definition subsystem 40 mayimpose a constraint upon one or more target reference points 34.Constraints may be purely user defined on a case-by-case basis, or maybe created by the system 20 through the implementation of user-definedprocessing rules. By imposing the constraint upon one or more targetreference points 34, the definition subsystem 40 can ensure that thetarget reference points 34 are adequate for generation of thegeometrical object 35. The definition subsystem 40 can impose anynumber, combination, or type of constraint. These restraints may includea requirement that a minimum number of target reference points 34 beidentified, that a minimum number of target reference points 34 not beco-linear, or that target reference points 34 be within or outside of aspecified area of an image.

The definition subsystem 40 generates the geometrical object 35 andcoordinates the geometrical object 35 with the second image 32, whichgeneration and coordination can be accomplished by any method known to aperson skilled in the art, including by transferring or copying thegeometrical object 35 to the second image 32. The definition subsystem40 can provide a plurality of controls for positioning the geometricalobject 35 within the second image 32. The controls may include any oneof or any combination of a control for shifting the geometrical object35 along a dimensional axis, a control for rotating the geometricalobject 35, a control for changing a magnification of the geometricalobject 35, a course position control, a fine position control, or anyother control helpful for a positioning of the geometrical object 35 inrelation to the second image 32.

The definition subsystem 40 can include a thumbnail image of thegeometrical object 35. In some embodiments, the definition subsystem 40can identify a plurality of positions of the geometrical object 35 inrelation to the second image 32. Those positions may include a grossposition and a fine position. The thumbnail image may be used toidentify gross or fine positions of the geometrical object 35 inrelation to the second image 32. The definition subsystem 40 canidentify a plurality of positions of the geometrical object in asubstantially similar and consistent manner. In some embodiments, thedefinition subsystem 40 adjusts the geometrical object 35 within thesecond image 32. The definition subsystem 40 may adjust a positioning ofthe geometrical object 35 within the second image 32.

The geometrical object 35 can be used to define template referencepoints 36. In one embodiment, vertices of the geometrical object 35correspond with template reference points 36 when the geometrical object35 is located within or about the second image 32. A positioning of thegeometrical object 35 in relation to the second image 32 positions thevertices or other relevant points of the geometrical object 35 so as todefine the template reference points 36. The definition subsystem 40 canprovide for an accuracy metric related to at least one of the templatereference points 36. The accuracy metric can identify a measurement ofaccuracy of a positioning of at least one of the template referencepoints 36 in relation to an estimated or predicted position of referencepoints within the second image 32.

The alignment system 20 can be applied to images involving two, three,or more dimensions. In some embodiments, an Affine transform heuristicis performed using various target reference points 34 and templatepoints 36. The Affine transform can eliminate shift, rotational, andmagnification differences between different images. In otherembodiments, different types of relationship-related heuristics may beused by the definition subsystem 40 and/or the combination subsystem 42.Other examples of heuristics known in the art that relate to potentialrelationships between images and/or points include a linear conformalheuristic, a projective heuristic, a polynomial heuristic, a piecewiselinear heuristic, and a locally weighted mean heuristic. The variousrelationship-related heuristics allow the system 20 to compare imagesand points that would otherwise not be in a format suitable for theestablishment of a relationship between the various images and/orpoints. In other words, the relationship-related heuristics such as theAffine transform heuristic are used to “compare apples to apples andoranges to oranges.”

B. Combination Subsystem

The combination subsystem 42 is responsible for creating the alignedimage 38 from the images and relationships maintained in the definitionsubsystem 40. The combination subsystem 42 includes the aligned image38. The combination subsystem 42 is configured to generate the alignedimage 38 from the first image 30, the second image 32, at least one ofthe target reference points 34, and at least one of the templatereference points 36. The generation of the aligned image 38 by thecombination subsystem 42 can be accomplished in a number of ways. Thecombination subsystem 42 may access the target reference points 34 andthe template reference points 36 from the definition subsystem 42. Thecombination subsystem 42 can generate an alignment calculation ordetermine a relationship between at least one of the target referencepoints 34 and at least one of the template reference points 36. Thecombination subsystem 42 can use an alignment calculation orrelationship to align the first image 30 and the second image 32. Inanother embodiment, the combination subsystem 42 uses locations of thetarget reference points 34 and the template reference points 36 togenerate the aligned image 38.

C. Interface Subsystem

An interface subsystem 44 can be included in the system 20 andconfigured to allow the system 20 to interact with users 22. Inputs mayreceived by the system 20 from the user 22 through the interfacesubsystem 44, and users 22 may view the outputs of the system 20 throughthe interface subsystem 44. Any data, command, or other itemunderstandable to the system 20 may be communicated to or from theinterface subsystem 44. In a preferred embodiment, the user 22 cancreate processing rules through the interface subsystem 44 that can beapplied to many different processing contexts in an ongoing basis. Theinterface subsystem 44 includes the interface 26 discussed above.

D. Detection Subsystem

A detection subsystem 45 can be configured to detect distortions, orother indications of a problem, relating to an aligned image 38. Thedetection subsystem 45 also allows a user 22 to check for distortions inan aligned image 38. Once a distortion has been detected, the detectionsubsystem 45 identifies the extent and nature of the distortion. Theuser 22 can use data provided by the detection subsystem 45 to check fora misalignment of a device or system that generated the first image 30or the second image 32. The detection subsystem 45 can be configured bya user 22 through the use of the interface subsystem 44.

III. Input/Output View

FIG. 3 is a flow diagram illustrating an example of how the systemreceives input and generates output. A computer program 50 residing on acomputer-readable medium receives user input 46 through an inputinterface 48 and provides output 54 to the user 22 through an outputinterface 52. The computer program 50 includes the target referencepoints 34, the geometrical shape 35; FIGS. 1-2, the first image 30, thesecond image 32, the template reference points 36, a third image, andthe interface 26. As previously discussed, the target reference points34 are associated with the first image 30. The computer program 50 cangenerate a geometrical object 35 or shape in a number of ways, includingby connecting at least a subset of the target reference points 34. Thegeometrical shape 35 can be any number or combination of any shape,including but not limited to a segment, line, ellipse, arc, polygon, andtriangle.

The input 46 may include a constraint imposed upon the target referencepoints 34 or the geometrical shape 35 by the computer program 50. Byimposing a constraint upon target reference points 34, the computerprogram 50 ensures that the target reference points 34 are adequate forgeneration of the geometrical shape 35. The system 20 can impose anynumber, combination, or type of constraint, including a requirement thata minimum number of target reference points 34 be identified, that aminimum number of target reference points 34 not be co-linear, or thattarget reference points 34 be within or without an area. In oneembodiment, the computer program 50 requires more than four targetreference points 34. The computer program 50 may identify a centroid ofthe geometrical shape 35.

The second image 32 can be configured to include the geometrical shape35. The geometrical shape 35 is generated by the computer program 50within the second image 32. The computer program 50 can accomplish ageneration of the geometrical shape 35 within the second image 32 in anumber of ways. For example, the computer program 50 may transfer orcopy the geometrical shape 35 from one image to another.

The computer program 50 provides for identifying the template referencepoints 36 or locations of the template reference points 36 in relationto the second image 32. In one embodiment, the template reference points36 can be identified by a positioning of the geometrical shape 35 inrelation to a second image 32, which positioning is provided for by thecomputer program 50. The computer program 50 provides for a number ofcontrols for positioning the geometrical shape 35 within the secondimage 32. The manipulation of the controls is a form of input 46. Thecontrols may include any one of or any combination of a shift control, arotation control, a magnification control, a course position control, afine position control, or any other control helpful for a positioning ofthe geometrical shape 35 in relation to a second image 32. The controlscan function in a number of modes, including a coarse mode and a finemode. The computer program 50 provides for positioning the geometricalshape 35 by shifting the geometrical shape 35 along a dimensional axis,rotating the geometrical shape 35, and changing a magnification of thegeometrical shape 35. A positioning of the geometrical shape 35 caninclude a coarse adjustment and a fine adjustment. The computer program50 is capable of identifying of plurality of positions of thegeometrical shape 35 in relation to the second image 32, including agross position and a fine position of the geometrical shape 35 inrelation to the second image 32. This identification can be performed ina substantially simultaneous manner. A thumbnail image of an areaadjacent to a vertex of the geometrical shape 35 can be provided by thecomputer program 50.

The computer program 50 can provide for an accuracy metric related to atleast one of the template reference points 36. The accuracy metric is aform of output 54. The accuracy metric can identify a measurement ofaccuracy of a positioning of at least one of the template referencepoints 36 in relation to an estimated or predicted position of referencepoints within the second image 32.

The third image (the aligned image 38) is created from the first image30, the second image 32, and a relationship between the target referencepoints 34 and the template reference points 36. The creation of thethird image by the computer program 50 can be accomplished in a numberof ways. The computer program 50 can generate an alignment calculationor determine a relationship between at least one of the target referencepoints 34 and at least one of the template reference points 36. Thecomputer program 50 can use an alignment calculation or relationship toalign the first image 30 and the second image 32. In another embodiment,the computer program 50 uses locations of the target reference points 34and the template reference points 36 to generate the third image.

The computer program 50 can be configured to detect distortions of thethird image. Once a distortion has been detected, the computer program50 can identify the extent and nature of the distortion. A user 22 canuse data generated by the computer program 50 to check for amisalignment of a device or system that generated the first image 30 orthe second image 32. The output 54 of the computer program 50 caninclude various distortion metrics, misalignment metrics, and otherforms of error metrics (collectively “accuracy metrics”).

The interface 26 of the computer program 50 is configured to receiveinput. The interface 26 can include an input interface 48 and an outputinterface 52. The input can include but is not limited to an instructionfor defining the target reference points 34 and a command forpositioning the geometrical shape 35 in relation to the second image 32.The computer program 50 can be configured to execute other operationsdisclosed herein or known to a person skilled in the art that arerelevant to the present invention.

IV. Process-Flow Views

A. Example 1

FIG. 4 is a flow diagram illustrating an example of facilitating apositioning of images and generating an aligned image according to thepositioned images. At 56, a relationship is defined between the variousimages to be aligned. At 57, the system 20 facilitates the positioningof the images in accordance with the previously defined relationship.For example, the template image 32 is positioned in relation the targetimage 30 and the target image 30 is positioned in relation to thetemplate image 32. At 58, the system generates the aligned image 38 inaccordance with the positioning performed at 57.

The system 20 can perform the three steps identified above in a widenumber of different ways. For example, the positioning of the images canbe facilitated by providing controls for the positioning of the templateimage in relation to the object image.

B. Example 2

FIG. 5 is a flow diagram illustrating an example of steps that an imagealignment system 20 may execute to generate the aligned image 38.

At 60, the system 20 receives input for defining the target referencepoints 34 associated with a first image 30. Once the input 46 isreceived, or as it is received, the system 20 can then at 62 generate atthe geometrical object 35. The input 46 may include a command. Thegeometrical object 35 can be generated in a variety of ways, such as byconnecting the target reference points 34. In the preferred embodiment,the system 20 may be configured to require that at least four targetreference points 34 be connected in generating the geometrical object35. The geometrical object 35 can take any form or shape that connectsthe target reference points 34, and each target reference point 34 is avertex or other defining feature of the geometrical object 35. In onecategory of embodiments, the geometrical object 35 is a polygon.

At 64, the system 20 imposes and checks a constraint against the targetreference points 34. If the target reference points 34 at 66 do not meetconstraints imposed by the system 20, the system 20 at 68 prompts andwaits for input changing or adding to definitions of the targetreference points 34. Once addition reference point data is received, thesystem 20 again generates a geometrical object 35 at 62 and checksconstraints against the target reference points 34 at 64. The system 20may repeat these steps until the target reference points 34 satisfyconstraints. Any type of a constraint can be imposed upon the targetreference points 34, including requiring enough target reference points34 to define a particular form of geometrical object 35. For example,the system 20 may require that at least four target reference points 34are defined. If more than two target reference points 34 are co-linear,the system 20 may require that additional target reference points 34 bedefined. The system 20 may use the geometrical object 35 to imposeconstraints upon the target reference points 34.

Once the target reference points 34 are deemed at 66 to meet theconstraints imposed by the system 20, the system 20 generates ageometrical object 35 within the first image 30 and regenerates thegeometrical object 35 in the second image 32 space at 70. Thegeometrical object 35 can be generated in the second image 32 space in anumber of ways, including transferring or copying the geometrical object35 from the first image 30 to the second image 32. The geometricalobject 35 can be represented by a set of connected points, a solidobject, a semi-transparent object, a transparent object, or any otherobject that can be used to represent a geometrical object 35. Any suchrepresentation can be displayed by the system 20.

The system 20 identifies the template reference points 36 based on aplacement of the geometrical object 35 in relation to the second image32. In one embodiment, the method of identifying the template referencepoints is providing controls for positioning at 71 the geometricalobject 35 in relation to the second image 32. A variety of controls canbe made available, including one of or a combination of controls forshifting the geometrical object 35 up, down, left, or right in relationto the second image 32, rotating the geometrical object 35 in relationto the second image 32, changing the magnification or size of thegeometrical object 35 in relation to the second image 32, moving thegeometrical object 35 through multiple dimensions, switching betweencoarse and fine positioning of the geometrical object 35, or any othercontrol that can be used to adjust the geometrical object 35 in relationto the second image 32. A command can be received as an input 46allowing for the positioning of the geometrical object 35 by at leastone of rotating the geometrical object 35, adjusting a magnification ofthe geometrical object 35, and shifting the geometrical object 35 alonga dimensional axis. A command may allow for coarse and fine adjustmentsof the geometrical object 35.

In one embodiment, the system 20 provides a thumbnail image to theinterface 26 for displaying an area proximate to at least one of thetemplate reference points 36. The thumbnail image can be configured toallow for a substantially simultaneous display of fine positioningdetail and coarse positioning detail, for example, by providing both aview of thumbnail images and a larger view of the geometrical object 35in relation to the second image 32 to the user 22 for simultaneousviewing.

The system 20 may provide for an accuracy metric or an accuracymeasurement detail for either a composite of the template referencepoints 36 or individually for at least one or more of the individualtemplate reference points 36. The system 20 may provide accuracy metricsby calculating a number of accuracy metrics. Accuracy metrics facilitatean optimal positioning of the geometrical object 35 within the secondimage 32. In one embodiment, the system 20 receives input commands froman interface or from the user 22 for positioning the geometrical object35 at 72 in relation to the second image 32. In one category ofembodiments, the system 20 adjusts a positioning of the geometricalobject 35 at 74 within the second image 32. This adjustment can be basedupon the accuracy metric. The system 20 may use a computer implementedprocess, such as a refinement heuristic, or any other image alignmenttool for adjusting a placement of the geometrical object 35 in relationto the second image 32.

In other embodiments, the locations of the template reference points 36can be determined in other ways. For example, the user 22 may define thetemplate reference points 36 by pointing and clicking on locationswithin the second image 32, or the template reference points 36 can bepredefined. Once locations of the template reference points 36 have beendetermined, the system 20 can determine a relationship at 78 between thetarget reference points 34 and the template reference points 36. Such arelationship can be a mathematical relationship and can be determined inany of a number of ways.

The system 20 at 80 generates the aligned image 38 from the first image30 and the second image 32. In one embodiment, the generation occurs bythe system producing the aligned image 38 from the first image 30, thesecond image 32, and a relationship between at least one of the targetreference points 34 in the first image 30 and at least one of thetemplate reference points 36 in the second image 32. The system 20 canuse an alignment calculation or a computer implemented combinationheuristic to generate the aligned image 38. Some such heuristics areknown in the prior art.

The system 20 can check at 82 for distortions of the aligned image 38.By checking for a distortion in the aligned image 38, the system 20 candetect a possible misalignment of a device used to generate the firstimage 30 or the second image 32. In one embodiment of the presentinvention, the system 20 checks for distortions in the aligned image 38by comparing the locations of the vertices of the geometrical object 35in relation to the second image 32 with defined or desired locations ofthe template reference points 36, which defined or desired points may beindicated by the user 22 of the system 20. This comparison ofdiscrepancies produces an alignment status. The system 20 analyzes thedegree and nature of any misalignment between locations of the verticesof the geometrical object 35 and defined locations of the templatereference points 36 to reveal information about the degree and nature ofany misalignment of an image generating device or system. Analyzingdistortions allows the system 20 or the user 22 to analyze the alignmentstatus of an image generation device.

C. Example 3

FIG. 6 is a flow diagram illustrating an example of steps that a user 22of an image alignment system 20 can perform through an access device 24and an interface 26 to generate an aligned image 38.

At 84, the user 22 selects or inputs images for alignment 84. The user22 can provide images to the system 20 in any form recognizable by thesystem 20, including digital representations of images. The user 22 at86 defines the target reference points 34 of a first image 30. Thetarget reference points 34 can be defined by pointing and clicking onlocations within the first image 30, by importing or selectingpredefined target reference points 34, or by any other wayunderstandable to the system 20.

The user 22 of the system 20 at 88 can initiate generation of thegeometrical object 35. The geometrical object 35 can be initiated in anumber of ways, including the defining the target reference points 34,defining a set number of the target reference points 34 that satisfyconstraints, submitting a specific instruction to the system 20 togenerate the geometrical object 35, or any other means by which the user22 may signal the system 20 to generate the geometrical object 35. Thesystem 20 can select an appropriate type of geometrical object 35 togenerate, or the user 22 may select a type of geometrical object 35 tobe generated. In one embodiment, the system 20 generates a geometricalobject 35 by connecting the target reference points 34.

A user determines a centroid 104 of a geometrical object 35. In analternative embodiment, the system 20 can determine and indicate thecentroid 104 of the geometrical object 35. A determination of thecentroid 104 is helpful for eliminating or at least mitigating errorsthat can occur in the image alignment system 20. By determining thecentroid 104 of the geometrical object 35, the user 22 can verify thatthe centroid 104 is near the center of a critical area of the firstimage 30. If the system 20 or the user 22 of the system 20 determinesthat the centroid 104 of the geometrical object 35 is not near enough toa critical area of the first image 30 as is desired, the user 22 canredefine the target reference points 34.

The user 22 of the system 20 at 92 initiates a transfer or copy of thegeometrical object 35 to the second image 32 space. The user 22 maysignal the system 20 to transfer the geometrical object 35 in any wayrecognizable by the system 20. One such way is by sending an instructionto the system 20 for generation of the geometrical object 35 in thesecond image 32. Upon receipt of an appropriate signal, the system 20transfers or copies the geometrical object 35 to the second image 32space.

Once the geometrical object 35 is transferred to the second image 32space, the user 22 positions the geometrical object 35 within the secondimage 32 space. The user 22 can use controls provided by the system 20or that are a part of the system 20 to position the geometrical object35. In one embodiment, the user 22 can shift the geometrical object 35up, down, left, or right in relation to the second image 32, rotate thegeometrical object 35 in relation to a second image 32, change themagnification or size of the geometrical object 35 in relation to thesecond image 32, move the geometrical object 35 through multipledimensions, switch between coarse and fine positioning of thegeometrical object 35, or execute any other control that can be used toadjust the geometrical object 35 in relation to the second image 32. Theuser 22 may use a thumbnail view or an accuracy metric to position thegeometrical object 35.

The user 22 of the system 20 initiates alignment of the first image 30and the second image 32. The user 22 may signal the system 20 totransfer the geometrical object 35 in any way recognizable by the system20. One such way is to send an instruction for alignment to the system20 via the access device 24 or the interface 26. Upon receipt of analignment signal, the system 20 generates the aligned image 38 from thefirst image 30 and the second image 32.

At 98, the user 22 of the system 20 checks for distortion of the alignedimage 38. In one embodiment, the user 22 determines and inputs to thesystem 20 desired locations of the template reference points 36 inrelation to the second image 32. The system 20 can analyze the desiredlocations and post-alignment locations of template reference points 34to discover information about any distortions in an aligned image 38.The system 20 may reveal to the user 22 any information pertaining to adistortion analysis.

V. Examples of Reference Points and Geometric Objects

FIG. 7A illustrates the target reference points 34 defined in relationto the first image 32. FIG. 7B illustrates the geometrical object 35connecting the target reference points 35. FIG. 7C illustrates anindication of the centroid 104 of the geometrical object 35. FIG. 7Dillustrates a transferred geometrical object 35 and the templatereference points 36 in relation to the second image 32.

FIG. 7A is a diagram illustrating one example of target reference points34 defined in relation to the first image 32. FIG. 7B is a diagramillustrating one example of a geometrical object 35 connecting targetreference points 34 associated with a first image 30. FIG. 7C is adiagram illustrating an example of a geometrical object 35 and acentroid 104 associated with that geometrical object 35. FIG. 7D is adiagram illustrating a transferred geometrical object 35 and varioustemplate reference points 36 positioned in relation to a second image32.

VI. Alternative Embodiments

The above description is intended to be illustrative and notrestrictive. Many embodiments and applications other than the examplesprovided would be apparent to those of skill in the art upon reading theabove description. The scope of the invention should be determined, notwith reference to the above description, but should instead bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled. It isanticipated and intended that future developments will occur in imagealignment systems and methods, and that the invention will beincorporated into such future embodiments.

1-49. (canceled)
 50. A method for aligning radiographic images,comprising: facilitating a positioning of a template image in relationto an object image; and generating an aligned image from a target imageand said object image according to said positioning of said templateimage in relation to said object image.
 51. The method of claim 50,wherein said positioning is facilitated by providing controls for saidpositioning of said reference image in relation to said object image.52. An apparatus for aligning images, comprising: a computer programtangibly embodied on a computer-readable medium, said computer programincluding instructions for: facilitating a positioning of a templateimage in relation to an object image; and generating an aligned imagefrom a target image and said object image according to said positioningof said template image in relation to said object image.
 53. Theapparatus of claim 52, wherein said positioning is facilitated byproviding controls for said positioning of said reference image inrelation to said object image.
 54. An apparatus for aligning images,comprising: a computer program tangibly embodied on a computer-readablemedium, said computer program including instructions for: receiving aninput for defining target reference points associated with a firstimage; generating a geometrical object by connecting at least four saidtarget reference points; identifying template reference points based ona placement of said geometrical object in relation to said second image;and producing an aligned image from said first image, said second image,and a relationship between at least one of said target reference pointsin said first image and at least one of said template reference pointsin said second image.
 55. The apparatus of claim 54, said computerprogram further including instructions for imposing a constraint on saidtarget reference points.
 56. The apparatus of claim 54, wherein saidinput includes a command.
 57. The apparatus of claim 56, wherein saidcommand allows for at least one of shifting said geometrical objectalong a dimensional axis, rotating said geometrical object, andadjusting a magnification of said geometrical object.
 58. The apparatusof claim 56, wherein said command allows for coarse adjustments and fineadjustments of said geometrical object.
 59. The apparatus of claim 54,said computer program further including instructions for calculating aplurality of accuracy metrics.
 60. The apparatus of claim 54, saidcomputer program further including instructions for displaying saidgeometrical object as one of a solid object, a semi-transparent object,and a transparent object.
 61. The apparatus of claim 54, said computerprogram further including instructions for displaying a thumbnail imageof an area proximate to at least one of said template reference points.62. The apparatus of claim 61, wherein said thumbnail image isconfigured to allow for a substantially simultaneous display of finepositioning detail and coarse positioning detail.
 63. The apparatus ofclaim 54, said computer program further including instructions forproviding an accuracy measurement detail for at least one of saidtemplate reference points or for a composite of said template referencepoints.
 64. The apparatus of claim 61, said computer program furtherincluding instructions for analyzing an alignment status of an imagegeneration device, wherein said alignment status is determined from adiscrepancy between said accuracy measurement detail and definedlocations for said template reference points.
 65. The apparatus of claim54, further comprising adjusting said positioning of said geometricalobject within said second image.